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ABSTRACT 



- ■ ■ -k Collecting time-on- cask data Is necessary in order to research 
methods for increasing student tim.-on-task and to provide £ e oife aek tt 
teachars on wnat teaching behavior, produce high levels of on-task 
behaviors in students. Instruments developed by the Detroit Public 
schools as part of a research contract with the National Institute of 
Education in the area of academic learning time are presented k this 
paper along with Information on training persons to use these 
instruments and suggestions for their application in various 'stings. 
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ASSESSING TIME-ON-TASK: MEASUREMENT PROBLEMS AND SOLUTIONS 

Introduction 

The Detroit Public School District is one of four sites under 
contract with the National Institute of Education to conduct research 
funded by Follow Through monies in the area of Academic Learning Time 



One of the major components of ALT is the concept of time-on-task, 
During the first year of Detroit's four-year contract, extensive planning 
activities were undertaken in the area of data collection. Instruments 
were developed to collect data on Students' on-task/of f-task status 
during mathematics and reading classes. The students involved in this 
project were in grades one through four. Instruments to collect data on 
teacher behaviors were also developed in order to provide feedback to 
teachers relative to which of their behaviors resulted in high on-task 
student behavior and which resulted in off-task student behavior. 
Simultaneous measures of teacher behavior and student on-task/of f-task 
behavior were required, 

Data collection procedures had to be simple enough so that all 
teachers involved in the project would feel comfortable using them and 
unobtrusive so as not to interrupt the normal classroom routine. Data 
from these instruments were used for evaluation purposes for the project i 
interim results to help steer participants through a series of interven- 
tions designed to increase their knowledge of classroom management 
strategies and Instructional strategies which research has shown to 
increase student on-task behavior and student achievement, and pre-pest 
data used to assess the success of the project over time. 
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This paper describes^ two instruments developed during this planning 
period and refined during the first year of project implementation. 
Training procedures for persons using the instruments are also described. 

Student Data Collection 

After reviewing the literature (Berish and Schulta, 1981; Floden and 
cer, 1981; Stallings, 1980), it was determined that observations made 
every two (2) minutes throughout a class period would give a valid 
"snapshot" of student behavior. Data were collected on every pupil in 
the room during the class period so that comparisons for individual 
students could be made in their on- task rates and correlations with 
achievement gains could be examined. Students in this project were quite 
young and classroom practices frequently involved regrouping pupils 
several times during a single instructional period (e.g., reading groups 
moved from seat work to oral reading and instruction). These constraints 
indicated that the data collection method employed must be flexible while 
recording data on every child every two (2) minutes. 

FIGURE 1 ABOUT HERE 
For students in the upper grades, a seating chart format was used. 
An example of this seating chart appears in Figure 1. On the seating 
chart, each student's name was recorded in a rectangle representing 
his/her location in the classroom. Sufficient space was available in the 
rectangle to record 25 codes, one every two minutes for each child. The 
codes were recorded in five rows of five codes each. The codes were; 
a plus sign (+) when the student was determined to be on-task, a minus 

sign (-) when the student was determined to be off -task, and a zero (0) 

i 
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when the coder was unable to determine the student 1 d status. For those 
students in the lower grades, a list of the children 1 s names was used and 
children wore name tags so that the coder was able to identify them, 
Coding took place adjacent to each student's name on the list using the 
same codes used on the seating chart, This procedure allowed coders ro 
keep track of children in classes where frequent regrouping of pupils 
made a seating chart impractical* 

Definitions of on-task and off- task behavior weie developed along 
with examples of each type of behavior in order to insure reliable data 
on students, Ou-task behavior was defined as students participating in 
the intended lesson which was related to either reading or mathematics * 
Examples of on-task behaviors included t participating in guided lessons, 
responding orally, engaged in a written assignment , engaged in a 
discussion that was relative to the lesson, and taking a test or quiz* 

Off-task behavior was defined as behaviors not related to the lesson 
or a lack of involvement on the part of students* Examples of off-task 
behaviors included: engaged in a social interaction, uninvolved in the 
lesson, exhibiting disruptive behavior, waiting for help, being 
disciplined, sharpening pencils, daydreaming, and roving about the room, 

Coders were instructed to attempt to code students either on- or 
off-task if at all possible. The zmto (0) code was used only when the 
coder was unable to observe the student for some reason- Students were 
coded based upon a snapshot approach I the coder ^swept 1 * the students , 
always following the same path, and recorded what he/she observed each 
student's status to be at the moment of the observation* Because coding 
must be completed very rapidly, no attempt was made to make cumulative 
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judgement, concerning the students' bthavior during the entire two-minute 
interval, Since each of the 25 observations represented only 4 percent 
of the total observation, errors resulting from thi* procedure were 
minimized. 

Classroom Data Collection 

Data on teacher behaviors occurring during the class period were 
also collected, One of she primary project goals was to increase 
teachers' use of ins true tionally appropriate behaviors while decreasing 
those teacher behaviors which caused students to move off-task or which 
research had shown did not produce positive results in terms of increased 
student learning. It was necessary to collect data on teacher behaviors 
concurrently with student data in order to link student behaviors with 
teacher behaviors during feedback sessions with teachers. 

In addition to data on specific teacher behaviors, data indicating 
the activities in which all members of the class were Involved ware 
important. Student data indicated only on-task/of f -task status. In 
order to document the activities occurring in the classroom while data 
were being collected, the teacher behavior instrument was designed to 
account for all activities occurring in the classroom during each two- 
minute coding period. Numbers of students involved in each activity were 
recorded in order to achieve this objective. A sample of the Classroom 
Observation Form appears In Figure 2, 

FIGURE 2 ABOUT HERE 
Four categories of data were recorded on this form: non-interactive 
instruction, interactive instruction, off-task, and organization. 
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Non-interactive instruction included silent reading and seat work. 
Interactive instruction included oral reading, instructional explanation, 
giving Erections, discuss/review assignments , and practice/drill. Off- 
task behaviors indicated the number of students off -task. Organization 
included management, distribute/collect materials, and transitions. 
Specific definitions of each of these terms appear in Figure 3. Each 
observation period consisted of two minutes. During this two-minute 
period, the coder first recorded data for each of the students in the 
class on the Student Observation Form, After all students had bean 
coded, the number of students involved in each of the activities on the 
Classroom Observation Form was recorded, Finally, the activity in which 
the teacher was directly involved was indicated by circling the number of 
students involved in that activity. 

FIGURE 3 ABOUT HERE 
As a guide to participant.;, target percents for each area were also 
indicated on the form. These targets were from Stallings (1980). She 
found that the most effective teachers used interactive instruction at 
least 50 percent of the time, used non-interactive instruction, at most, 
35 percent of the time, used 15 percent or less of the time for organiza- 
tional activities, and usest 5 percent or less of the instructional time 
in off-task behaviors. In order for participants to compute their own 
percentages based upon observation data, the number of observation 
periods in which the teacher was involved in each of the four areas was 
recorded and converted to a percent, This number was indicated on the 
observation form by counting how many circled numbers were in each 
activity. 
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Co der Training 

The instruments described above were designed for use by elementary 
teachers conducting classroom observations of their peers. The instru- 
ments were field tested in classrooms at all levels, K-12, and were found 
to give valid and reliable data on student on- and off-task behaviors and 
classroom activities. The instruments were also used by graduate 
students in education to collect data in project classrooms which were 
used as baseline information before project implementation and as a final 
observation for pre-post comparisons for project evaluation purposes 
during each year of project implementation. 

Training observers required that they become familiar with the 
operational definitions of the behaviors to be observed ,nd that they be 
given an opportunity to familiarize themselves with the instruments 
before attempting to collect any data, 

Once coders felt comfortable with the definitions and the location 
of the items on the observation forms, the group was given an opportunity 
to observe video tapes of classes in session which could be stopped so 
that discussion of various events could take place in terms of how they 
should be coded on the forms. In the Detroit project, video tapes from 
the Georgia Teacher Evaluation Project were used for this phase of the 
training. Concensus was reached by the coders as to how various 
situations should be coded in a relatively short period of time. 

The next step was to send coders into actual classrooms in pairs. 
Each coder in the pair coded the students and the classroom behaviors 
simultaneously but 'without knowledge of the codes used by his/her 
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partner. Following the coding purled, a da-briefing session where the 
pair exchanged tha results of their coding and discussed any discrepan- 
cies was helpful in improving inter-coder reliabilities* Several 
practice sessions were provided so that coders felt comfortable with the 
forms and the procedures involved* 

Most individuals were able to learn to use the forms and collect 
reliable and valid data in two days, The initial apprehension expressed 
by most coders the project trained was that the time constraints involved 
were unrealistic; i.e., two minutes was not enough time to collect data 
on all students and to record a snapshot of the classroom. Following 
training, nearly ail found that they had more than enough time to collect 
and record the required data, and that the two minute time period was 
much longer than they first believed* 

Summary and Recommendations 

t Collecting reliable and valid data on students and teachers related 
to time-on-task can be accomplished using a single observer in a class- 
room if sufficient planning is undertaken and if the observer is properly 
trained . 

- f ue results of the data collection are useful to teachers who have 
been observed, The data give teachers feedback relative to their 
behaviors and teaching practices and information on every student in 
their classroom over an entire class period* The procedures outlined 
above have been used successfully to collect data in classrooms from 
kindergarten through grade twelve. The behaviors observed are a subset 
of all possible behaviors which could be observed in a classroom, It is 
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strongly suggested that teacher—s who are to be observed, have input into 
which behaviors are to be obser— ved and that opportunities for help be 
offered to teachers tyto fall be low or above target in any given 
behavioral area. It is also suggested that student data be used by 
teachers to target ingtruetion «md to examine the difficulty level and 
appropriateness of learning ttasWcs assigned to students who are habitually 
off-task. If the process of collecting data in the area of tima-on-task 
is viewed as beneficial to all involved, it can result in improved 
instruction. If, how eV er, it i» viewed as punitive by teachers, the 
results in changed behavior wtim be minimal. 
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Figure 3 

PEER TEACHERS AS MIRRORS AND MONITORS 
Classroom Observation Form Activities 



ACTIVITY 



Silent Reading 



DESCRIPTION 



Seat Work 



Oral Reading 



Instructional 
Explanation 



Giving Directions 



Discuss/Review 
Assignment 



Practice/Drill 



Students are reading silently to themselves as a 
group activity or are working on individual 
assignments. No writing. 

One or more students is/are writing papers, doing 
computation, or involved in any other silent 
written work related to the lesson* 

One or more students is/are reading a section from 
a play aloud or reading a book for the class or 
reading group to hear. Oral reading is usually 
not done in unison; generally, students take turns 
reading sequential sections from a book, The 
taacher or the student (s) can also read aloud 
while the rest of the class follow along in their 
own texts. 

An adult is informing some grouping of students 
about a subject. Academic discussion or slow- 
paced quention/answer session takes place 
regarding lecture material, assignments , or 
problems , 

An adult is explaining an activity, the procedures 
to be followed, the amount of work to be finished, 
or rewards for completing the assignment. The 
discussion is not focusing en the academic content, 
but on the information that students need to carry 
out the assignment (or discussing grades)* 

One or more students is/are receiving information 
or feedback on work they have completed, or are 
being evaluated on their work preparatory to 
continuing the assignment. 

One or more students is/are verbally involved in 
reinforcing, repetitive, or rote work. This 
activity must be differentiated from seat work, 
Students writing verbal material, as in dictation, 
are also coded Practice/Drill, 



16 



Figure 3 (Continued) 

PEER TEACHERS AS MIRRORS AND MONITORS 
Classroom Observation Form Activities 



ACTIVITY 



Students Off-Task 



DESCRIPTION 



Management 



Distribute/Collect 
Materials 



Transitions 



One or more students or teacher and students are 
interacting about work or subjects other th^n 
class-related material, or students are net 
involved in any activity, are arriving or leaving, 
or moving about the room, (See list of off-task 
behaviors ) 

Taking attendance, making/receiviui announcements, 
regrouping, forming lines, discipline, collecting 
money, etc. 

Teacher and/or students are involved in passing 
out papers, putting away materials, preparing to 
leave, or preparing or checking materials. 

Changing from one activity to another* 
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